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A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 
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earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)E3 Responsive to communication(s) filed on 15 December 2003 . , 
2a)D This action is FINAL.. 2b)g] This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-48 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) [3 Claim(s) 1-48 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)13 The drawing(s) filed on 15 October 2003 is/are: a)E3 accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1 .121 (d). ■ 
1 !)□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)DAII b)Q Some * c)Q None of: 

1. D Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3-D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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Detailed Action 

Background 

1. This action is responsive to the following communications: Application 
filed on 12/15/2003. 

2. Claims 1-48 are pending in this case. Claims 1, 17, 29, and 41 are in 
independent form. 

Information Disclosure Statement 

3. The information disclosure statements (IDS) submitted on 12/15/2003, 
1/24/2005, 5/3/2005, 5/16/2005, 8/9/2005, 1/27/2006, 5/5/2006 have been 
acknowledged and considered by the examiner. Initial copies of the form PT01449 
are included in this office action. 

Abstract of the Disclosure 

4. The abstract of the disclosure is objected to because it exceeds 150 words. 
Correction is required. See MPEP § 608.01(b). 

It is important that the abstract not exceed 150 words in length since the space 
provided for the abstract on the computer tape used by the printer is limited. 
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Claim Rejections • 35 USC § 102 



5. The following is a quotation of the appropriate paragraphs of 35 
U.S.C. 102 that form the basis for the rejections under this section made in this 
Office action: 

(e) the invention was described in (I) an application for patent, 
published under section 122(b), by another filed in the United 
States before the invention by the applicant for patent or (2) a 
patent granted on an application for patent by another filed in 
the United States before the invention by the applicant for 
patent, except that an international application filed under the 
treaty defined in section 351(a) shall have the effects for 
purposes of this subsection of an application filed in the United 
States only if the international application designated the United . 
States and was published under Article 21(2) of such treaty in 
the English language. 

6. Claims 1-46 are rejected under 35 U.S.C. 102(e) as being anticipated by 
Edwin A. Suominen (US Pre-Grant Publication 2003/0055655 A l) (hereinafter 
Suominen). 

As to independent claim 1, Suominen describe(s): A method comprising' 
receiving electronic ink input ("...freehand input to generate and edit text data...," para. 
[0041]); generating a list of machine-generated text candidates based on the electronic 
ink input ("...text that has been generated (erroneously) by a message recognizer...," 
para. [0130]), the list including a first machine-generated text candidate and 
alternative machine-generated text candidates ("...a list of alternative text comparable 
to the selection is provided...," para. [0132]); converting the electronic ink input to the 
first machine-generated text candidate ("...shown by way of example in TABLE IV and 
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TABLE V...," para. [0132]); displaying the first machine-generated text candidate 
("...displayed list of alternative text para. [0133]); receiving speech input ("...voice 
input...," para. [0093]); converting the speech input to second machine-generated text 
("...Speech recognizer 162 generates text data responsive to voice input from user 240 
and provides the text data to editor 168...," para. [0045]), wherein the second machine- 
generated text is one of the alternative machine-generated text candidates and the list 
of machine-generated text candidates functions as a dictionary used for converting the 
speech input ("...text segments chosen from the corpus...," para. [0172]); and replacing 
the first machine-generated text candidate with the second machine-generated text 
("...desired modifications to text data generated by speech recognizer 162 in accordance 
with editing commands communicated by freehand input and interpreted by tablet 
interface 164....," para. [0097]). 

As to dependent claim 2, which depends from claim 1, Suominen further 
disclose(s): The method of claim 1, wherein the first machine -generated text candidate 
is a word ("...a word ...," para. [0004]). 

As to dependent claim 3, which depends from claim 1, Suominen further 
disclose(s): The method of claim 1, wherein the first machine -generated text candidate 
is a portion of a word ("...text segments are referred to herein as "stop segments," and 
may include more or less text than a single word...," para. [0148]). 

As to dependent claim 4, which depends from claim 1, Suominen further 
disclose(s): The method according to claim 1, further comprising receiving input 
selecting the first machine-generated text candidate prior to receiving the speech input 
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("... the selected text segment is modified responsive to the character input....," para. 
[0134]). 

As to dependent claim 5, which depends from claim 4, Suominen further 
disclose(s): The method according to claim 4, wherein the selecting includes touching a 
user input device to a digitizer screen at a location corresponding to the first machine- 
generated text candidate ("...by stylus tap somewhere on the text...," para. [0092]). 

As to dependent claim 6, which depends from claim 4, Suominen further 
disclose(s): The method according to claim 4, wherein the first machine -generated text 
candidate is a group of words or part of a word ("...a phrase of several words.;.," para. 
[0131]). 

As to dependent claim 7, which depends from claim 1, Suominen further 
disclose(s): The method according to claim 1, further including displaying the list of 
machine-generated text candidates prior to receiving the speech input ("...the list of 
alternative text is updated, based on character data communicated after initiation of 
the correction mode...," para. [0133]). 

As to dependent claim 8, which depends from claim 7, Suominen further 
disclose(s): The method according to claim 7, wherein said step of displaying the 
alternative machine-generated text candidates further includes displaying the 
alternative machine-generated text candidates in the list in an order based on a 
confidence level that each alternative machine-generated text candidate corresponds to 
the electronic ink input ("... The list includes text segments that each received a high 
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score (but not the maximum score) during hypothesis testing of the message recognizer. 
Two lists of alternative text of the type that may be produced.../' para. [0132]). 

As to dependent claim 9, which depends from claim 1, Suominen further 
disclose(s): The method according to claim 1, wherein the alternative machine- 
generated text candidates include machine-generated text candidates based on the 
electronic ink input generated by a handwriting recognition engine ("...handwriting 
recognition engine..., 11 para. [0042]). 

As to dependent claim 10, which depends from claim 9, Suominen further 
disclose(s): The method according to claim 9, wherein the alternative machine - 
generated text candidates include machine-generated text candidates based on the 
electronic ink input generated in accordance with a statistical language model 
("...Probability Pr(w.sub.iw.sub.j.vertline.V) may be computed (at least as an estimate) 
by any suitable statistical technique. For example, probability 
Pr(w.sub.iw.sub.j.vertline.V) may be computed for a given combination of text segments 
w.sub.i and w.sub.j based on the total number of occurrence of segment wi in matrix Z, 
divided by the total number of occurrences of all segments in matrix Z. The probability 
of shared cluster occurrence of text segments w.sub.i and w.sub.j may be estimated as 
the ratio between (l) the number of times text segments w.sub.i and w.sub.j can be 
paired together within semantic clusters of the corpus and (2) the total number of text 
segments in the corpus....," para. [0173]). 

As to dependent claim 11, which depends from claim 10, Suominen further 
disclose(s): The method according to claim 10, further comprising displaying the 
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machine-generated text candidates generated by the handwriting recognition engine 
and subsequently displaying the machine-generated text candidates generated in 
accordance with the statistical language model ("...Language model 350 includes a 
syntactic model 352 and a semantic model 354. Syntactic model 352 provides a set of a 
priori probabilities based on a local word context, and may be a conventional N-gram 
model. Semantic model 354 provides probabilities based on semantic relationships 
without regard to the particular syntax used to express those semantic relationships, 
para. [0146]). 

As to dependent claim 12, which depends from claim 11, Suominen further 
disclose(s): The method according to claim 11, further comprising receiving input 
requesting the display of the machine-generated text candidates generated in 
accordance with the statistical language model while displaying the machine-generated 
text candidates generated by the handwriting recognition engine ("...Handwriting 
recognizer 314 then derives units of language (again, strings of text data) from the 
units of handwriting in accordance with language model 350....," para. [0145]). 

As to dependent claim 13, which depends from claim 1, Suominen further 
disclose(s): The method according to claim 1, wherein the alternative machine- 
generated' text candidates include text candidates based on the electronic ink input 
generated by a statistical language model ("...Probability Pr(w.sub.iw.sub.j.vertline.V) 
may be computed (at least as an estimate) by any suitable statistical technique. For 
example, probability Pr(w.sub.iw.sub.j.vertline.V) may be computed for a given 
combination of text segments w.sub.i and w.sub.j based on the total number of 
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occurrence of segment wi in matrix Z, divided by the total number of occurrences of all 
segments in matrix Z. The probability of shared cluster occurrence of text segments 
w.sub.i and w.sub.j may be estimated as the ratio between (l) the number of times text 
segments w.sub.i and w.sub.j can be paired together within semantic clusters of the 
corpus and (2) the total number of text segments in the corpus... .," para. [0173]). 

As to dependent claim 14, which depends from claim 1, Suominen further 
disclose(s): The method according to claim 1, wherein the step of converting the speech 
input to the second machine-generated text includes determining if the speech input 
corresponds to one of the alternative machine-generated text candidates ("...Speech 
recognizer 312 generates text data in accordance with acoustic model 356 and shared 
language model 350....," para. [0126])(emphasis added); and converting the speech input 
to the corresponding alternative machine-generated text candidate when the speech 
input corresponds to the alternative machine-generated text candidate ("...speech 
recognizer such as a computer executing the NATURALLY SPEAKING software, the 
modifications to the selected text segment are entered when the user selects an "OK" 
control of a displayed user interface dialog. ...," para. [0134]). 

As to dependent claim 15, which depends from claim 1, Suominen further 
disclose(s): The method according to claim 1, wherein further comprising the step of 
receiving an input confirming that the second machine-generated text candidate should 
replace the first machine -generated text candidate prior to performing said step of 
replacing (when the user selects an "OK" control of a displayed user interface dialog. 
...," para. [0134]). 
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As to dependent claim 16, this claim differs from claim 1 only in that it is directed to 
a product defined by the process of claim 1. Accordingly, this claim is rejected for the same 
reasons set forth in the treatment of claim 1, above. 

As to claims 17-28, these claims differ from claims 1-16, respectively, only in that 
they are directed to objects whereas claims 1-16 are directed to text. Since "text" is a 
type of object ("The machine-generated objects may correspond to words, lines, and/or 
other groupings of machine-generated text.", see applicants disclosure at para. [27]), 
claims 17-28, are rejected for the same reasons set forth in the treatment of claims 1* 
16, respectively. 

As to claims 29-40, these system claims differ from claims 1-15 in that they are 
directed to a system for carrying out the methods defined by claims 1-15. Accordingly, 
claims 29*40 are rejected for the same reasons set forth in the treatment of claims 1-15 
above. 

As to claims 41-48, these system claims differ from claims 29, 31, 34, and 36-40 
in that they are directed to "objects" whereas claims 29, 31, 34, and 36-40 are directed 
to "text." Since "text" is a type of object ("The machine-generated objects may 
correspond to words, lines, and/or other groupings of machine-generated text.", see 
applicants disclosure at para. [27]), therefore, claims 41-48, are rejected for the same 
reasons set forth in the treatment of claims 29, 31, 34, and 36-40. 
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Conclusion 

7. Although not relied upon, the following prior art is made of record because 
it considered pertinent to applicant's disclosure: 



Suominen, Edwin A. US 20030055655 A1 Text processing system 

Bangalore, Srinivas et al. US 20040119754 A1 Context-sensitive interface widgets for multi-modal dialog 

systems 

Bangalore, Srinivas et al. US 20040122674 A1 Context-sensitive interface widgets for multi-modal dialog 

systems 

Suominen, Edwin A. US 20050171783 A1 Message recognition using shared language model 
Bushey, Robert R. et al. US 20050183032 A1 System and method for managing recognition errors in a 

multiple dialog state environment 
Ditzik; Richard Joseph US 6167376 A Computer system with integrated telephony, handwriting and 

speech recognition functions 



8. Any inquiry concerning this communication or earlier communications 
from the Examiner should be directed to Samir Termanini whose telephone number is 
(571) 270-1047. The Examiner can normally be reached from 9 A.M. to 4 P.M., Monday 
through Friday (excluding alternating Fridays). 

If attempts to reach the Examiner by telephone are unsuccessful, the Examiner's 
supervisor, Stephen S. Hong can be reached on (571) 272-4124. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from either Private PAIR or Public PAIR. Status information 
for unpublished applications is available through Private PAIR only. For more information 
about the PAIR system, see http7/pair-direct.uspto.gov. Should you have questions on 
access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866* 
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217-9197 (toll-free). If you would like assistance from a USPTO Customer Service 
Representative or access to the automated information system, call 800-786-9199 (IN USA 
OR CANADA) or 571-272-1000. 





STEPHEN HONG 
SUPERVISORY PATENT EXAMINER 



Samir Termanini 
Patent Examiner 
Art Unit 2178 



